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DETAILED ACTION 

Information Disclosure Statement 

1 . The Examiner has considered the references listed in the Information Disclosure 
Statement dated 08/06/2004. A copy of the Information Disclosure Statement is 
attached to this office action. 

Claim Rejections - 35 USC §112 

The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 13 and 27 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. In these claims the term CTI is undefined. 

3. Claim 30 is rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. In this claim the term SQL is undefined. 



4. 



Claim Objections 
All relevant objects are withdrawn as being satisfied. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1 , 2, 4, 8, 9, 11-14, 16, 17, 19, 23, 24, and 26-28 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Clements et al. ("Phonetically Searching 
Applied to On-line Distance Learning Modules," IEEE DSP Workshop, October 13-16, 
2002), hereinafter referred to as Clements, in view of Glowny et al. (U.S. Patent 
Application Publication 2001/0040942 A1 ), hereinafter referred to as Glowny. 

Regarding claims 1 and 16, Clements teaches a phonetic searching technique 
that includes the following steps: 

• defining a phrase to use for searching (p. 2, col. 2, 1f2); 

• defining a minimum confidence level for searching (p. 3, col. 1, H's 1 and 2, 
ignore results below 90% confidence) ; 

• searching said set of audio segments for said phrase (p. 2, § High-Speed 
Phonetic Searching; p. 1, introduction, Infusion creates an index ...); 

• producing a set of results of all occurrences of the phrase within the set of audio 
segments and the confidence that a given occurrence is a match for the search phrase 
(p. 3, col. 1, H's 1 and 2, most likely candidates listed first). 
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Clements also teaches the use of metadata as an aid to searching (p. 1 , 
introduction, Infusion creates an index), but Clements does not specifically teach 
"identifying a set of said audio segments based on intrinsic data associated with said 
audio segments." However, the examiner contends that this concept was well known in 
the art, as taught by Glowny. 

In the same field of endeavor, Glowny discloses a method for recording and 
storing telephone call information. Glowny also supports the ability to recall and 
playback a recorded call that was stored in pieces (1J[0044] the reassembled call 
represents a set of audio data connected by metadata). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements by specifically providing the 
features, as taught by Glowny, because it is well known in the art at the time of 
invention for the purpose of saving the searcher time in locating a call (Glowny, U's 
[0002] and [0003]). 

Regarding claims 2 and 17, Clements in view of Glowny teach everything 
claimed, as applied above (see claims 1 and 16, respectively). In addition, Clements 
teaches: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, U 2, 
words or phrases), 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speech Phonetic Searching), 
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• said step of producing includes producing a set of results of all occurrences of 
the plurality of phrases identified in a specified sequential order within the audio 
segments with said minimum confidence that a given occurrence within said audio 
segments is a match for a corresponding one of said plurality of search phrases (p. 2, 
1T4 through p. 3, fl2). 

Regarding claims 4 and 19, Clements in view of Glowny teach everything 
claimed, as applied above (see claims 1 and 16). In addition, Clements teaches the 
following: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, H2), 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speech Phonetic Searching), 

• said step of producing includes producing a set of results of all occurrences of 
the plurality of phrases identified in a specified temporal relationship within the audio 
segments with said minimum confidence that a given occurrence within said audio 
segments is a match for a corresponding one of said plurality of search phrases (p. 2, 
col. 2, temporal operators, p. 3, col. 1, H's 1 and 2, § Integration Issues with Infusion). 

Regarding claims 8 and 23, Clements in view of Glowny teaches everything 
claimed, as applied above (see claims 1 and 16). In addition Clements teaches: 

• said step of defining a phrase includes defining a plurality of phrases (p. 2, col. 2, 
1J2, query term is specified as text containing one or more phrases), 
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• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speed Phonetic Searching), 

• and said step of producing includes producing a set of results of all occurrences 
of the plurality of phrases identified in a specified temporal relationship within the audio 
segments with said minimum confidence that a given occurrence within said audio 
segments is a match for a corresponding one of said plurality of search phrases (p. 2, 
col. 2, temporal operators, p. 3, col. 1, ConfidenceJLevel, § Integration Issues with 
Infusion). 

Regarding claims 9 and 24, Clements in view of Glowny teaches everything 
claimed, as applied above (see claims 8 and 23). In addition, Clements teaches "said 
temporal relationship is with respect to said phrases" (p. 2, col. 2, representing two 
phrases spoken within 5 seconds of each other). 

Regarding claims 11 and 26, Clements in view of Glowny teaches everything 
claimed, as applied above (see claims 1 and 16). Clements also teaches "the step of 
identifying said set of audio segments comprises a step of contstraining said set of 
audio segments to ones of said audio segments selected for processing based on said 
intrinsic data prior to performing said searching step" (§ High-Speed Phonetic 
Searching, ^'s 1 and 2, process the input speech producing the phonetic search track), 
integration Issues with Infusion, also see rejection of claims 1 and 16). 
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Regarding claim 12, Clements in view of Glowny teaches everything claimed, as 
applied above (see claim 1). But Clements does not specifically teach "said intrinsic 
data comprises metadata." However, the examiner contends that this concept was well 
known in the art, as taught by Glowny. 

In the same field of endeavor, Glowny discloses a method for recording and 
storing telephone call information. Glowny also teaches the use of additional data (i.e., 
intrinsic data) along with the voice recordings flf's [0003] and [0044], use of metadata is 
indicated). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Glowny by specifically 
providing additional data along with recording, as taught by Glowny, since additional 
information associated with a voice file can aid in searching by assembling a call 
(Glowny, U's [0002] and [0003]). 

Regarding claims 13 and 27, Clements in view of Glowny teaches everything 
claimed, as applied above (see claims 1 and 16), but Clements does not specifically 
teach "said intrinsic data comprises CTI data." However, the examiner contends that 
this concept was well known in the art, as taught by Glowny. 

In the same field of endeavor, discloses a method for recording and storing 
telephone call information. Glowny also teaches the use of additional CTI data along 
with the voice recordings fll's [0003] and [0044]). 
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Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Glowny by specifically 
providing additional data along with recording, as taught by Glowny, since additional 
information associated with a voice file can aid in searching by assembling a call 
(Glowny, f s [0002] and [0003]). 

Regarding claims 14 and 28, Clements in view of Glowny in view of Glowny 
teaches everything claimed, as applied above (see claims 13 and 27). But Clements 
does not specifically teach "said CTI data selected from the set consisting of (i) called 
number (DNIS) and, calling number (AN1), and (iii) Agent ID." However, the examiner 
contends that this concept was well known in the art, as taught by Glowny. 

Glowny further discloses the use of telephone number, caller ID and agent ID 
number (fl's [0003] and [0044]). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Glowny by specifically 
providing additional data along with recording, as taught by Glowny, since additional 
information associated with a voice file can aid in searching by assembling a call fll's 
[0002] and [0003]). 

6. Claims 3, 5, 6, 18, 20 and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Clements in view of Glowny and further in view of Frakes et al. 
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("Information Retrieval/Data Structures & Algorithms," Prentice Hall, 1992), hereinafter 
referred to as Frakes. 

Regarding claims 3 and 18, Clements in view of Glowny teach everything 
claimed, as applied above (see claims 1 and 16). In addition, Clements teaches: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, 1J2), 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speed Phonetic Searching), 

• and said step of producing includes producing a set of results of all audio 
segments including (i) at least one occurrence of a selected required one of the plurality 
of phrases (p. 3, col. 1, f s 1 and 2). 

Clements teaches the use of Boolean operations for queries (abstract), but 
Clements does not specifically teach "(ii) non-occurrences of at least one selected 
forbidden one of said plurality of phrases to be excluded from within the audio 
segments, said occurrence and non-occurrence determined with respect to said 
minimum confidence that a given occurrence within said audio segments is a match for 
a corresponding one of said plurality of search phrases." However, the examiner 
contends that this concept was well known in the art, as taught by Frakes. 

In the same field of endeavor, Frakes teaches basic techniques of information 
retrieval. Frakes's teachings include the use of the Boolean expression "not 
information" to find documents that do not contain information (p. 266), which 
correspond to "selected forbidden one of said plurality of phrases." 
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Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Glowny by specifically 
providing the "not" Boolean operator, as taught by Frakes, since it was well-know in the 
art that such an operation is useful during information retrieval tasks (Frakes is a 
textbook published in 1992 with widely known techniques). 

Regarding claims 5 and 20, Clements in view of Glowny teaches everything 
claimed, as applied above (see claims 1 and 16). In addition, Clements teaches the 
following: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, P), 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speed Phonetic Searching), 

Clements also teaches the use of Boolean operations for queries (abstract) and 
the use of temporal operators (p. 2, col. 2, 1J2), but Clements does not specifically teach 
"said step of producing includes producing a set of results of all audio segments lacking 
occurrences of the plurality of phrases identified in a specified temporal relationship 
within the audio segments with said minimum confidence that a given occurrence within 
said audio segments is a match for a corresponding one of said plurality of search 
phrases." However, the examiner contends that this concept was well known in the art, 
as taught by Frakes. 

In the same field of endeavor, Frakes teaches basic techniques of information 
retrieval. Frakes' teachings include the use of the Boolean expression "not information" 
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to find documents that do not contain information (p. 266), which correspond to 
"selected forbidden one of said plurality of phrases." 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Glowny by specifically 
providing the "not" Boolean operator, as taught by Frakes, since it was well-know in the 
art that such an operation is useful during information retrieval tasks (Frakes is a 
textbook published in 1992 with widely known techniques). 

Regarding claims 6 and 21, Clements in view of Glowny in view of Frakes 
teaches everything claimed, as applied above (see claims 5 and 20). In addition, 
Clements teaches "said temporal relationship is with respect to said phrases" (p. 2, col. 
2, 1J2, representing two phrases spoken within 5 seconds of each other). 

7. Claims 7 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Clements in view of Glowny and Frakes and further in view of Li et al. ("MOQL: A 
Multimedia Object Query Language," Third International Workshop on Multimedia 
Information Systems, Como, Italy, Sept. 1996), hereinafter referred to as Li. 

Regarding claims 7 and 22, Clements in view of Glowny and Frakes teaches 
everything claimed, as applied above (see claims 5 and 20). In addition, Clements 
teaches the use of temporal operators, but Clements does not specifically teach "said 
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temporal relationship is with respect to said audio segment." However, the examiner 
contends that this concept was well known in the art, as taught by Li. 

In the same field of endeavor, Li teaches the use of a multimedia object query 
language that includes the use of temporal functions. Li's teachings include a 
discussion of an anchored specification of time (i.e., a the temporal relationship is with 
respect to the segment) (§3.2). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Glowny and Frakes, as 
taught by Li, since this type of temporal operator supports typical multimedia queries (Li, 
§3.2). 



8. Claims 10 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Clements in view in view of Glowny and further in view of Li. 

Regarding claims 10 and 25, Clements in view of Glowny teaches everything 
claimed, as applied above (see claims 8 and 23). In addition, Clements teaches the use 
of temporal operators, but Clements does not specifically teach "said temporal 
relationship is with respect to said audio segment." However, the examiner contends 
that this concept was well known in the art, as taught by Li. 

In the same field of endeavor, Li teaches the use of a multimedia object query 
language that includes the use of temporal functions. Li's teachings include a 
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discussion of an anchored specification of time (i.e., a the temporal relationship is with 
respect to the segment) (§3.2). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Glowny, as taught by Li, 
since this type of temporal operator supports typical multimedia queries (Li, §3.2). 

9. Claims 15, 29 and 30 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Glowny in view of Clements. 

Regarding claim 15, Glowny teaches a method for recording and storing 
telephone call information. Glowny's method includes the following steps: 

• connecting a plurality of calls to at least one customer service representative 
(Fig. 1, items 130 and 160, Fig. 2, items 100, 230, U's [0035] through [0037]); 

• recording audio segments from each of said plurality of calls (Fig. 1 , item 145, 
Fig. 2, item 180, H [0037]); 

• identifying a set of said audio segments based on intrinsic data associated with 
said calls fll[0044] accessing and reassembling the segments of a call using metadata); 

• In addition, Glown teaches search and retrieval of recoded information (U's 
[0003] and [0009]), but Glowny does not specifically teach the following steps: 

a) defining a phrase to use for searching; 

b) defining a minimum confidence level for searching; 

c) searching said set of audio segment for said phrase; and 
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d) producing a set of results of all occurrences of the phrase within the audio 
segments and the confidence that a given occurrence is a match for the search 
phrase. 

However, the examiner contends that steps a) through d) were well known in the 
art, as taught by Clements. 

In the same field of endeavor, Clements teaches a phonetic searching technique 
that includes steps a) through d), see rejection of claim 1 , above. 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Glowny by specifically providing the steps a) 
through d), as taught by Clements, since such an approach is an efficient and high- 
speed technique for locating audio information (Clements, p. 1, Introduction and p. 
2, High-Speed Phonetic Searching, 1J1). 

Regarding claim 29, Glowny discloses a contact center that includes the 
following: 

• a switch configured to connect each of a plurality of calls to a customer service 
representative workstation (Fig. 2, items 100 and 230, U 0035); 

• a memory connected to said switch and configured to record audio segments 
from each of said plurality of calls (Fig. 1, item 145, Fig. 2, item 180). 

• a supervisory terminal configured to ... (ii) identify a set of said audio segments 
from ones of said plurality of calls selected by said supervisory terminal based on 
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intrinsic data associated with resprective one of said calls flj[0044] from a 
workstation accessing and reassembling the segments of a call using metadata); 
Glowny also teaches the use of a workstation (supervisory terminal) that can 

browse (search) the recorded information (Fig. 1 , item 160, U 0047), but Glowny does 

not specifically teach the following features: 

a) a supervisory terminal configured to (i) define a phrase to use for searching and 
a minimum confidence level for searching and 

b) a search engine connected to said supervisory terminal and to said memory for 
searching said set of audio segment for said phrase; and 

c) a display connected to said search engine and configured to produce a set of 
results of all occurrences of the phrase within the audio segments and the 
confidence that a given occurrence is a match for the search phrase. 

However, the examiner contends that steps a) through c) were well known in the art, as 
taught by Clements. 

In the same field of endeavor, Clements teaches a phonetic searching technique 
that includes the following: a) the ability to define a search phrase and a confidence 
level (p. 2, col. 2, 1J2; p. 3, col. 1, U's 1 and 2); b) a search engine with necessary 
memory to perform a search (§ High-Speed Phonetic Searching); and c) a necessary 
display for displaying results (p. 3, U's 1 and 2, enumerated results). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Glowny by specifically providing the steps a) 
through c), as taught by Clements, since such an approach is an efficient and high- 
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speed technique for locating audio information (Clements, p. 1, Introduction ffl, and p. 
2, High-Speed Phonetic Searching, 1J1). 

Regarding claim 30, Glowny discloses a method for recording and storing 
telephone call information includes the following: 

• storing audio segments in a speech repository (Fig. 1 , items 145 and 140, U's 
[0037] and [0305]); 

• storing information regarding each of the audio segments in a database (Fig. 1 , 
item 155, U's [0037] and [0305], metadata is stored with the audio recordings). 

In addition, Glowny teaches the search and retrieval of the information stored in a 
database fl|[0032]) using techniques that are compatible with SQL fl|[0225]), which 
corresponds to "establishing a search criteria including speech and SQL criteria for 
locating spoken words or phrases...", but Glowny does not specifically teach the 
following: 

a) ... for locating spoken words or phrases in said audio segment using speech 
recognition technology; 

b) searching said set of audio segments and said database in accordance with said 
search criteria; 

c) providing a report based on said search. 

However, the examiner contends that steps a) through c) were well known in the art, as 
taught by Clements. 
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In the same field of endeavor, Clements teaches a phonetic searching technique 
that includes the following: a) the ability to search for spoken phrases using speech 
recognition techniques (p. 2, col. 2, fl's 1 and 2); b) a search engine (§High-Speed 
Phonetic Searching, U 2; Fig. 1 , PAT file is a database of preprocessed audio; 
introduction, Infusion supports use of metadata); and c) a necessary report for 
displaying results (p. 3, fl's 1 and 2, enumerated results). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Glowny by specifically providing the steps a) 
through c), as taught by Clements, since such an approach is an efficient and high- 
speed technique for locating audio information (see p. 1, Introduction and p. 2, High- 
Speed Phonetic Searching, 1f1). 



Response to Arguments 

10. Applicant's arguments filed 02/02/2005 have been fully considered but they are 
not persuasive. 



1 1 . Applicant asserts on page 5: 

Thus, while Glowny uses metadata including CTI data to locate a targeted 
telephone call, it fails to describe or suggest using such data to select 
some subset of audio segments that are then subject to being searched to 
determine if one or more contain certain specified content. In particular, 
Glowny alone or in combination with Clements fails to describe or suggest 
the subject matter of the independent claims as presently amended 
including: ... [repetition of claim limitations for claims 1 , 15, 16, 29 and 30] 
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The Examiner directs the Applicant to the above corresponding rejections of 
these claims. In summary, Glowny can reassemble a call stored in segments (a subset 
of the audio data stored) using metadata fll[0044]), and Clements performs searching 
on phonetic search tracks (p. 2, §High-Speech Phonetic Searching) and is compatible 
with metadata (p. 1, introduction, Infusion creates an index that links text 
descriptors...). The combination of Clements and Glowny teach the selection of a 
subset of audio segments for subsequent searching. 

1 2. Applicant further asserts beginning on page 1 6: 

The rejection of the claims is further traversed and believed to be improper 
for failure of the applied references to suggest or otherwise provide 
motivation for making the asserted combination. While the reasoning set 
forth in the Office Action asserts that additional information associated 
with a voice file can aid searching, merely realizing that a combination 
provides certain advantages using Applicants' claims as a template 
amounts to no more than impermissible hindsight. ... 

In response to applicant's argument that the examiner's conclusion of 
obviousness is based upon improper hindsight reasoning, it must be recognized that 
any judgment on obviousness is in a sense necessarily a reconstruction based upon 
hindsight reasoning. But so long as it takes into account only knowledge which was 
within the level of ordinary skill at the time the claimed invention was made, and does 
not include knowledge gleaned only from the applicant's disclosure, such a 
reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 
1 971 ). Furthermore, in the motivation given for the rejections of amended claims 1 and 
1 6, Glowny indicates that the ability to combine the audio data is an aid to the searcher 
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(Glowny, U's [0002] and [0003]) and furthermore Clements supports the use of 
metadata during search (Clements, introduction). 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to V. Paul Harper whose telephone number is (571) 272- 
7605. The examiner can normally be reached on M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571 ) 272-7602. The fax phone 
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number for the organization where this application or proceeding is assigned is 703- 
872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



V. Paul Harper 
Patent Examiner 
Art Unit 2654 





